Split head invalidation for consumer batching in pointer rings

ABSTRACT

A split head invalidation system includes a first memory including a ring buffer, a second memory, and a processor in communication with the first memory. The processor includes a consumer processor and a producer processor. The consumer processor is configured to maintain a head and tail pointer, detect a request to copy a memory entry from the ring buffer, and consume the memory entry. Consuming the memory entry includes iteratively testing a value associated with the memory entry in a slot indicated by the head pointer, retrieving the respective memory entry from the slot, and advancing the head pointer to the next slot until reaching a threshold quantity of slots. Additionally, the consumer processor is configured to invalidate each slot from the head pointer to the tail pointer after reaching the threshold quantity.

BACKGROUND

Computer systems may routinely copy memory entries from one memory to adifferent memory. For example, while forwarding incoming networkingtraffic to a physical or virtual machine, packets may be received andlater copied to another memory location. Processors may executeinstructions to read, write, and copy memory entries, such as packetaddresses to forward networking traffic to different machines. Forexample, memory entries may be temporarily stored in ring buffers on afirst-in-first-out basis before being copied to the memory associatedwith a different machine. Specifically, incoming networking traffic maybe stored on a ring buffer and later copied to virtual machine memory.

SUMMARY

The present disclosure provides new and innovative systems and methodsfor reverse slot invalidation for pointer rings. In an example, a systemincludes a first memory including a ring buffer having a plurality ofslots, a second memory, and at least one processor in communication withthe first memory. The at least one processor includes a consumerprocessor and a producer processor. The consumer processor is configuredto maintain a head pointer and a tail pointer, detect a request to copyat least one memory entry from the ring buffer to the second memory, andconsume the at least one memory entry from the ring buffer. Consumingthe at least one memory entry includes testing a respective valueassociated with a respective memory entry in a first slot of theplurality of slots, where the first slot is indicated by the headpointer, and responsive to receiving a valid value from the test,retrieving the respective memory entry from the first slot.Additionally, consuming the at least one memory entry includes advancingthe head pointer to a second slot while the location of the tail pointerremains unchanged, iteratively repeating the testing, retrieving, andadvancing for each successive slot that the head pointer advances tountil reaching a threshold quantity of slots. Then, responsive to aquantity of slots reaching the threshold quantity of slots, the consumerprocessor is configured to invalidate each slot between the head pointerand the tail pointer.

In an example, a method includes maintaining, by a consumer processor, ahead pointer and a tail pointer. Additionally, the consumer processordetects a request to copy at least one memory entry from a ring bufferhaving a plurality of slots, and consumes the at least one memory entryfrom the ring buffer. Consuming the at least one memory entry includestesting a respective value associated with a respective memory entry ina first slot of the plurality of slots, where the first slot isindicated by the head pointer, and responsive to receiving a valid valuefrom the test, retrieving the respective memory entry from the firstslot. Additionally, consuming the at least one memory entry includesadvancing the head pointer to a second slot while the location of thetail pointer remains unchanged, iteratively repeating the testing,retrieving, and advancing for each successive slot that the head pointeradvances to until reaching a threshold quantity of slots. Then,responsive to a quantity of slots reaching the threshold quantity ofslots, the consumer processor invalidates each slot between the headpointer and the tail pointer.

In an example, a non-transitory machine-readable medium stores code,which when executed by a consumer processor, is configured to maintain ahead pointer and a tail pointer, detect a request to copy a at least onememory entry from a ring buffer having a plurality of slots, andconsume, by the consumer processor, the at least one memory entry fromthe ring buffer. Consuming the at least one memory entry includestesting a first value associated with a respective memory entry in afirst slot of the plurality of slots, where the first slot is indicatedby the head pointer. Consuming the at least one memory entry furtherincludes responsive to receiving a valid value from the test, retrievingthe respective memory entry from the first slot. Additionally, consumingthe at least one memory entry includes advancing the head pointer to asecond slot while the location of the tail pointer remains unchanged,iteratively repeating the testing, retrieving and advancing for eachsuccessive slot that the head pointer advances to until reaching athreshold quantity of slots. Additionally, the non-transitorymachine-readable medium is configured to invalidate each slot betweenthe head pointer and the tail pointer responsive to a quantity of slotsreaching the threshold quantity of slots.

Additional features and advantages of the disclosed method and apparatusare described in, and will be apparent from, the following DetailedDescription and the Figures. The features and advantages describedherein are not all-inclusive and, in particular, many additionalfeatures and advantages will be apparent to one of ordinary skill in theart in view of the figures and description. Moreover, it should be notedthat the language used in the specification has been principallyselected for readability and instructional purposes, and not to limitthe scope of the inventive subject matter.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates a block diagram of an example computing systemaccording to an example embodiment of the present disclosure.

FIG. 2A illustrates a block diagram of an example memory ring accordingto an example embodiment of the present disclosure.

FIG. 2B illustrates a block diagram of an example memory ring accordingto an example embodiment of the present disclosure.

FIG. 3 illustrates a block diagram of split head invalidation forconsumer batching of an example memory ring.

FIG. 4 illustrates a flowchart of an example process for split headinvalidation for consumer batching according to an example embodiment ofthe present disclosure.

FIGS. 5A, 5B, and 5C illustrate a flow diagram of an example process forsplit head invalidation for consumer batching in pointer rings accordingto an example embodiment of the present disclosure.

FIG. 6 illustrates a block diagram of an example split head invalidationsystem according to an example embodiment of the present disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

Techniques are disclosed for reverse slot invalidation for pointer ringswhen copying data from one memory location (e.g., ring buffer) to adifferent memory. For example, the techniques disclosed may be used whenforwarding incoming network traffic to a virtual machine by ahypervisor, which may include receiving a packet from a networkinterface controller (NIC) in hypervisor memory and copying the packetto virtual machine memory. The act of receiving the data (e.g., packets)and copying the data may be executed on the same processor (e.g., CPU),however, parallelizing the actions on separate processors or separateprocessor cores may provide significant performance advantages.Currently, techniques of parallelizing the receiving and copying mayinvolve a first CPU receiving an interrupt from a NIC, receiving packetsfrom the NIC, and adding the packets to a list in a shared memory. Asecond CPU runs a thread to retrieve the packets from the list and copythe packets to VM memory. Because the parallel process utilizes twoCPUs, the process adds additional overhead because adding and removingpackets on one list requires cross-CPU communication through sharedmemory. Traditionally, a linked list or a producer/consumer ring wasused without much added performance as false cache sharing typicallyoutweighed the benefits of parallelism.

Additionally, other approaches such as a typical circular buffer designoften creates cache line bounces between the two CPUs. For example, aring data structure (e.g., an array with a plurality of slots) may beused with a producer processor and a consumer processor (or producer CPUand consumer CPU). In the disclosed implementation, an interrupthandling processor or producer processor may maintain a producerpointer. The producer pointer may store data, such as the address ofeach packet received at the slot addressed by the producer pointer. Theproducer processor may increment the pointer to address the next slot,thereby wrapping around at the end of the array. To avoid overruns,before storing the data (e.g., the address of each packet and/orpacket), the producer processor may test the value in each slot. If thevalue is valid (e.g., non-NULL), the data is not stored in the list andmay be discarded. A data copying processor or consumer processor maymaintain a consumer pointer. The consumer processor may test the valuepointed to by the consumer pointer. If the value is invalid (e.g.,NULL), then the array is empty and the consumer processor may stop andwait for more packets. If the value is valid (e.g., non-NULL), theconsumer processor may retrieve the data, such as a packet address.Then, the consumer processor may invalidate the slot (e.g., write NULLinto the slot) and may advance the consumer pointer to the next slot.The retrieved data may be copied to a second memory location (e.g.,virtual machine memory).

Invalidating a slot (e.g., storing a NULL value in a slot)advantageously allows reuse of the slot for additional data (e.g.,forwarding a new packet). However, this data structure may experienceperformance bottlenecks. For example, when a driver is highly optimized,copying may be slower than receiving packets. Thus, writing an invalid(e.g., NULL) value by the copying thread or consumer processor may causecache invalidation messages to be sent to the interrupt handling threador producer processor. Additionally, the consumer processor may beblocked from consuming additional memory entries until each of theinvalidation messages is received. As a result, the ring may be full fora large part of the time, and as each packet slot is invalidated by theconsumer processor, the packet slot may be immediately made valid againby the interrupt handling processor or producer processor, which maycause a cache line to bounce between the processors and results in asignificant slowdown. Due to the bottleneck and resulting slowdown fromcache line bounces and waiting for cache invalidation messages to bereceived, the consumer processor may be unable to get ahead of theproducer processor resulting in cache line bounces for each dataoperation in the ring buffer.

A proposed solution to the cache line bounces and the blockage ofconsumer processor described above was to load a network interfacecontroller (NIC) driver in a virtual machine as a pass-through to avoidthe need for packet forwarding. However, this approach breaks migrationand overcommit, which may cause the host or computer system to crash.Additionally, even though consuming multiple memory entries or packetsat once may cause a single invalidation, such batching may requirewaiting for multiple packets to become available through receivecoalescing or rx coalescing, which may increase latency and add overheadfor running timers on the host. For example, rx coalescing may wait apredetermined amount of time before raising an rx interrupt after amemory entry or packet has been received. Similarly, rx coalescing mayinvolve waiting for a specified number of memory entries or packets toqueue up before raising an rx interrupt. Further, this proposed solutionmay require an updated application program interface (API) to enable theconsumer processor to consume multiple packets at a time, which may onlybe beneficial if multiple packets are available. For example, theproposed solution of loading a NIC driver in a virtual machine as apass-through (e.g., with rx coalescing) may cause delays while waitingfor additional packets and may result in additional overhead as a resultof using a complex consumer processor.

Another proposed solution to the cache line bounces described above wasto reserve extra slots in the ring, which would remain unused to spaceout the producer thread and consumer thread such that they were notaccessing the same cache line. However, ring size is often designed inpowers of two to allow optimization. If a ring with 128 active slots isneeded, then by adding unused slots, the implemented ring would bedesigned with 256 slots (e.g., next power of two above 128 slots). Forexample, if 128 active slots will be utilized, and 10 slots will remainunused, then the ring needs a minimum of 138 slots. Due to the commonpractice of increasing ring buffer size by a factor of two, the nextlogical ring buffer size is 256 slots (e.g., next largest size up from128 slots). However, larger rings are problematic because the processorskeep watch over the ring, and a larger ring puts more pressure on cache.Moreover, only actively using, for example, 128 slots of 256 slots isnot an efficient use of hardware resources.

As described in the various example embodiments disclosed herein, toreduce the frequency of cache line bounces and prevent slowdown whilewaiting for cache invalidation messages to be received, the consumerprocessor maintains a head pointer and a tail pointer and detects arequest to copy one or more memory entries (e.g., data such as packetaddresses) from a ring buffer. The consumer processor consumes thememory entries by testing values associated with the respective memoryentries in successive slots in the ring buffer. If the consumerprocessor receives a valid value from the test, the consumer processorretrieves the respective memory entry from the slot. After retrievingthe memory entry, the consumer processor advances to the head pointer tothe next successive slot and continues to test and retrieve memoryentries from successive slots until the head pointer advances to athreshold quantity of slots (thereby retrieving a threshold quantity ofmemory entries). In an example, the consumer processor may copy therespective memory entries to a second memory and may track a quantity ofslots that the head pointer advances to. Once the quantity of slotsreaches the threshold quantity of slots, the consumer processorinvalidates each slot between the head pointer and the tail pointer, forexample by invaliding slots from the head pointer all the way back toand including the tail pointer. For example, the consumer processor maywalk the ring buffer backwards from the current location of the headpointer to the first slot (e.g., original location of the head pointer)and may invalidate each of the slots while walking backwards. Due toinvalidating slots in groups, the producer processor is held back andonly starts adding new entries in the ring after the consumer processorhas made progress. Thus, the consumer processor may advantageouslyretrieve a group of packets elsewhere in the ring (e.g., a few slotsahead of the producer processor) such that retrieving a single packet nolonger causes cache line bounces between the producer processor andconsumer processor even when the ring is full or almost full.Additionally, invalidating the slots in groups may advantageously reducethe quantity of cache invalidation messages to be sent to the interrupthandling thread or producer processor, thereby reducing the occurrenceof the consumer processor being blocked while waiting for the producerprocessor to receive each of the cache invalidation messages. Thus,cache line bounces and consumer processor blockage events areadvantageously reduced and performance is improved. For example, theabove consumer processor modification may improve packet throughput byapproximately ten percent or more.

Instead of the cache line constantly bouncing between the producerprocessor and consumer processor, the consumer processor retrieves andinvalidates a threshold quantity of memory entries at a time, whichadvantageously allows the consumer processor to retrieve multiple memoryentries before the cache line bounces to the producer processor andbefore sending an invalidation messages to the producer processor,thereby improving performance and throughput without doubling the ringsize as described above.

FIG. 1 depicts a high-level component diagram of an example computingsystem 100 in accordance with one or more aspects of the presentdisclosure. The computing system 100 may include an operating system(e.g., host OS 186), one or more virtual machines (VM 170A-B), nodes(e.g., nodes 110A-B), a consumer processor 124, a producer processor128, and memory 134 including a ring buffer 138. Ring buffer 138 may bea data structure using a single, fixed-size buffer as if it wereconnected end-to-end (e.g., in a ring). In an example, the ring buffer138 may be a first-in-first-out (FIFO) data structure. For example,memory entries such as packet addresses may be written into andretrieved from the ring buffer 138. Additionally, the ring buffer 138may have a plurality of slots, which may store memory entries. The slotsmay be tracked by pointers or indices, such as a head pointer and a tailpointer. In an example, the consumer processor 124 may maintain the headpointer and the tail pointer.

Virtual machines 170A-B may include a guest OS, guest memory, a virtualCPU (VCPU), virtual memory devices (VIVID), and virtual input/outputdevices (VI/O). For example, virtual machine 170A may include guest OS196A, guest memory or virtual machine memory 195A, a virtual CPU 190A, avirtual memory devices 192A, and virtual input/output device 194A.Virtual machine memory 195A may include one or more memory pages.Similarly, virtual machine 170B may include guest OS 196B, virtualmachine memory 195B, a virtual CPU 190B, a virtual memory devices 192B,and virtual input/output device 194B. Virtual machine memory 195B mayinclude one or more memory pages.

The computing system 100 may also include a hypervisor 180 and hostmemory 194. Hypervisor 180 may manage host memory 184 for the hostoperating system 186 as well as memory allocated to the virtual machines170A-B and guest operating systems 196A-B such as guest memory orvirtual machine memory 195A-B provided to guest OS 196A-B. Host memory184 and virtual machine memory 195A-B may be divided into a plurality ofmemory pages that are managed by the hypervisor 180. Virtual machinememory 195A-B allocated to the guest OS 196A-B may be mapped from hostmemory 184 such that when a guest application 198A-D uses or accesses amemory page of virtual machine memory 195A-B, the guest application198A-D is actually using or accessing host memory 184.

In an example, a virtual machine 170A may execute a guest operatingsystem 196A and run applications 198A-B which may utilize the underlyingVCPU 190A, VIVID 192A, and VI/O device 194A. One or more applications198A-B may be running on a virtual machine 170A under the respectiveguest operating system 196A. A virtual machine (e.g., VM 170A-B, asillustrated in FIG. 1) may run on any type of dependent, independent,compatible, and/or incompatible applications on the underlying hardwareand OS. In an example, applications (e.g., App 198A-B) run on a virtualmachine 170A may be dependent on the underlying hardware and/or OS 186.In another example embodiment, applications 198A-B run on a virtualmachine 170A may be independent of the underlying hardware and/or OS186. For example, applications 198A-B run on a first virtual machine170A may be dependent on the underlying hardware and/or OS 186 whileapplications (e.g., application 198C-D) run on a second virtual machine(e.g., VM 170B) are independent of the underlying hardware and/or OS186A. Additionally, applications 198A-B run on a virtual machine 170Amay be compatible with the underlying hardware and/or OS 186. In anexample embodiment, applications 198A-B run on a virtual machine 170Amay be incompatible with the underlying hardware and/or OS 186. Forexample, applications 198A-B run on one virtual machine 170A may becompatible with the underlying hardware and/or OS 186A whileapplications 198C-D run on another virtual machine 170B are incompatiblewith the underlying hardware and/or OS 186. In an example embodiment, adevice may be implemented as a virtual machine (e.g., virtual machine170A-B).

The computer system 100 may include one or more nodes 110A-B. Each node110A-B may in turn include one or more physical processors (e.g., CPU120A-C) communicatively coupled to memory devices (e.g., MD 130A-C) andinput/output devices (e.g., I/O 140A-B). Each node 110A-B may be acomputer, such as a physical machine and may include a device, such ashardware device. In an example, a hardware device may include a networkdevice (e.g., a network adapter or any other component that connects acomputer to a computer network), a peripheral component interconnect(PCI) device, storage devices, disk drives, sound or video adaptors,photo/video cameras, printer devices, keyboards, displays, etc. Virtualmachines 170A-B may be provisioned on the same host or node (e.g., node110A) or different nodes. For example, VM 170A and VM 170B may both beprovisioned on node 110A. Alternatively, VM 170A may be provided on node110A while VM 170B is provisioned on node 110B.

In an example, consumer processor 124 and producer processor 128 may beone of the other processor(s) illustrated in FIG. 1, such as a CPU(e.g., CPU 120A-C) on node 110A-B. Similarly, ring buffer 138 may bestored in a memory device, and may be one of the other memory(s)illustrated in FIG. 1, such as MD 130A-C on node 110A-B. Additionally,consumer processor 124 and producer processor 128 may be different coreson the same physical processor. The consumer processor 124 may beconfigured to retrieve memory entries from the ring buffer 138 bytesting values of slots to determine whether the slots contain a memoryentry or an invalid value. The consumer processor 124 may also beconfigured to invalidate slots in batches, for example, by walking thering buffer 138 backwards after retrieving memory entries from athreshold quantity of slots. The consumer processor 124 may invalidate aslot by overwriting the current memory entry in the slot with a “0”value. By invalidating the threshold quantity of slots in batches, theconsumer processor prevents cache line bounces after each memory entryretrieval, thereby improving performance and throughput. Similarly, theconsumer processor sends an invalidation message for a batch or group ofslots instead of for each individual slot, thereby reducing the time andoccurrences of that the consumer processor is blocked while waiting forthe producer processor to receive the cache invalidation messages.

As used herein, physical processor or processor 120A-C, 124, and 128refers to a device capable of executing instructions encodingarithmetic, logical, and/or I/O operations. In one illustrative example,a processor may follow Von Neumann architectural model and may includean arithmetic logic unit (ALU), a control unit, and a plurality ofregisters. In a further aspect, a processor may be a single coreprocessor which is typically capable of executing one instruction at atime (or process a single pipeline of instructions), or a multi-coreprocessor which may simultaneously execute multiple instructions. Inanother aspect, a processor may be implemented as a single integratedcircuit, two or more integrated circuits, or may be a component of amulti-chip module (e.g., in which individual microprocessor dies areincluded in a single integrated circuit package and hence share a singlesocket). A processor may also be referred to as a central processingunit (CPU).

As discussed herein, a memory device 130A-C refers to a volatile ornon-volatile memory device, such as RAM, ROM, EEPROM, or any otherdevice capable of storing data. As discussed herein, I/O device 140A-Crefers to a device capable of providing an interface between one or moreprocessor pins and an external device capable of inputting and/oroutputting binary data.

Processors 120A-C may be interconnected using a variety of techniques,ranging from a point-to-point processor interconnect, to a system areanetwork, such as an Ethernet-based network. Local connections withineach node, including the connections between a processor 120A-C and amemory device 130A-C may be provided by one or more local buses ofsuitable architecture, for example, peripheral component interconnect(PCI).

FIGS. 2A and 2B illustrate block diagrams of example ring buffers 200Aand 200B. For example, FIG. 2A illustrates a linear bufferimplementation of ring buffer 200A while FIG. 2B illustrates the “ring”structure of ring buffer 200B. It should be appreciated that ringbuffers 200A-B may be the same actual memory structure illustrated intwo different ways. Ring buffers 200A-B may in be located in cacheablememory, such as L1 cache if on the same physical processor but ondifferent CPU cores. In another example, the ring buffer 200A-B may beon a different level of cache other than L1. Additionally, ring buffers200A-B may include a plurality of slots (e.g., slots 210A-N). Forexample, slots 210A-N may correspond to Slot_1 to Slot_n respectively.Each slot may include a memory entry, such as a data packet, a packetaddress, or the like. Additionally, slots may be empty or may include aninvalid value, such as “0”. For example, a slot with a memory address of“0” or a NULL value may be used to indicate an empty slot or invalidslot. Valid slots may include a memory entry, such as a data packet or apacket address.

FIG. 3 illustrates a block diagram of split head invalidation forconsumer batching of an example memory ring, such as ring buffer 138 or200A-B. For example, memory ring 300 is illustrated as 300A-E, whichrepresents different states of memory ring 300 at different points intime. Memory ring or ring buffer 300 may include six slots (e.g., slots210A-F). In an example, each slot may include a memory entry, such as apacket address P_1 to P_5 or an invalid value, such as a NULL value. Asillustrated in ring buffer 300A, a first slot (e.g., slot 210A) includesan invalid value, slot 210B includes packet address P_1, slot 210Cincludes packet address P_2, slot 210D includes packet address P_3, slot210E includes packet address P_4, and slot 210F includes packet addressP_5. Packet addresses may be written into the memory ring or ring buffer300 by a producer processor, such as producer processor 128.Additionally, indices or pointers may indicate specific slots in ringbuffer 300. For example, a head pointer 310A may designate or indicate afirst slot (e.g., slot 210B). The head pointer 310A may be advanced tosuccessive slots until reaching a threshold quantity of slots (e.g., twoslots). In an example, the head pointer may be an index, such as aregistry. A consumer processor 124 may retrieve memory entries, such aspacket addresses from a plurality of slots (e.g., slots 210B-D) afterthe head pointer 310A has advanced a threshold quantity of slots (e.g.,two slots). In the illustrated example, the head pointer 310A advancesto a second slot (e.g., slot 210C), which is illustrated as head pointer310B in ring buffer 300B. Additionally, the head pointer 310B mayadvance to a third slot (e.g., slot 210D), which is illustrated as headpointer 310C in ring buffer 300C. After retrieving the packet addresses,the consumer processor 124 may invalidate the group of slots (e.g.,slots 210B-D).

As illustrated by ring buffers 300C, the consumer processor 124 mayinvalidate each slot between the head pointer 310C and the tail pointer320A (e.g., from the head pointer 310C to the tail pointer 320A). Forexample, the consumer processor 124 may walk the ring 300 backwards andinvalidate each slot from the fourth slot (e.g., slot 210D) indicated bythe head pointer 310C to the second slot (e.g., slot 210B) indicated bythe tail pointer 320A. As illustrated in FIG. 3, invalidating each slotbetween the head pointer 310C and the tail pointer 320A may includeinvalidating slots from the head pointer 310C up to the tail pointer320A (e.g., slot 210C) as well as the slot indicated by the tail pointer320A (e.g., slot 201B). For example, the consumer processor 124 mayinvalidate slot 210C by writing or storing a NULL value into the slot.Similarly, the consumer processor 124 may invalidate slot 210B bywriting a NULL value into the slot 210B. After each slot (e.g., slot210B-C) has been invalidated, the tail pointer 320A may be advanced tothe same slot as the head pointer 310C, which is illustrated in ringbuffer 300D with the tail pointer 320B and head pointer 310C positionedat slot 210D with packet address P_3. Similar to the split headinvalidation for consumer batching described above for slots 210B-C, theconsumer processor 124 may proceed consuming additional memory entries,such as packet addresses or data packets, from the slot currentlyindicated by the head pointer 310C (e.g., slot 210D) to an additionalthreshold quantity of slots. Once the consumer has invalidated theslots, the producer processor 128 may store additional memory entries,such as packet address P_6 into ring buffer 300. For example, theproducer processor 128 may store P_6 and P_7 in slots 210A and 210Brespectively after the consumer processor 124 is done accessing thecache line for slots 210A and 210B, which advantageously reduces thefrequency of cache line bounces between the consumer processor 124 andproducer processor 128. For example, the consumer processor 124retrieves multiple memory entries before invalidating any of theentries, thereby holding back the producer processor 128 andadvantageously preventing the frequency of cache line bounces withoutincreasing ring size. Additionally, by invalidating groups or batches ofslots, cache invalidation messages are sent to the producer processor128 less frequently, thereby reducing the time and occurrences that theconsumer processor is blocked while waiting for the producer processor128 to receive the cache invalidation messages.

As illustrated in FIG. 3, the threshold quantity of slots is two,however, other threshold quantities may be used. For example, thethreshold quantity may be three, four, ten, etc. In an example, thethreshold quantity may be associated with the size of the ring buffer138. For example, the threshold quantity may be a fraction of the sizeof the ring buffer 138 (e.g., 1/16 of a ring, ⅛ of a ring, or ¼ of aring). In an example, if the threshold quantity is 1/16 of a ring with128 slots, then the threshold quantity may be 8 slots. Similarly, thethreshold quantity may also be related to the size and or quantity ofcache lines. For example, the threshold quantity may be calculated suchthat the threshold quantity of slots occupy an integer number of cachelines (e.g., 1, 2, 3). For example, if a slot size is 8 bytes and acache line is 64 bytes, a threshold quantity of slots may be 8, suchthat the threshold quantity of slots occupies one cache line. Similarly,the threshold quantity of slots may be 16 such that the thresholdquantity of slots occupies two cache lines. Additionally, the thresholdquantity of slots may be related to the quantity of cache lines used bythe ring buffer 138. For example, if the ring buffer 138 uses four cachelines (e.g., 32 slot ring where each slot is 8 bytes and four cachelines of 64 bytes), then the threshold quantity of slots may be thequantity of slots dedicated to a single cache line (e.g., 8 slots),which may advantageously prevent cache line bounces between the producerprocessor 128 and the consumer processor 124 because each processor maybe accessing different cache lines. For example, after the consumerprocessor 124 consumes and invalidates slots in a first cache line, itmay start consuming additional slots in a different cache line (e.g.,second cache line) as the producer processor 128 is producing memoryentries or data packets to slots in the first cache line, therebyallowing the producer processor 128 and the consumer processor 124 tosimultaneously execute memory operations in separate cache lines.

FIG. 4 illustrates a flowchart of an example method 400 for split headinvalidation for consumer batching according to an example embodiment ofthe present disclosure. Although the example method 400 is describedwith reference to the flowchart illustrated in FIG. 4, it will beappreciated that many other methods of performing the acts associatedwith the method 400 may be used. For example, the order of some of theblocks may be changed, certain blocks may be combined with other blocks,and some of the blocks described are optional. The method 400 may beperformed by processing logic that may comprise hardware (circuitry,dedicated logic, etc.), software, or a combination of both.

The example method 400 includes maintaining a head pointer and a tailpointer (block 410). For example, a consumer processor 124 may maintaina head pointer 310 and a tail pointer 320 in memory (e.g., in the samecache line). In an example, the head pointer 310 and the tail pointer320 are maintained in memory, which is private to the consumer processor124 and kept separate from memory accessed by the producer processor128. Maintaining a consumer index (e.g., head pointer 310 and/or tailpointer 320) in private memory advantageously reduces or avoids thecache line bounces inherent in standard circular buffer designs.Additionally, by using two consumer indices (e.g., head pointer 310 andtail pointer 320) in private memory, the amount of cache line bouncingmay advantageously be reduced further. Then, the consumer processor maydetect a request to copy at least one memory entry from a ring bufferhaving a plurality of slots (block 420). For example, a consumerprocessor 124 may detect a request to copy memory entries from a ringbuffer 138 to a second memory, such as virtual machine memory 195A. Inan example, the consumer processor 124 may detect a request after aproducer processor 128 adds additional memory entries to the ring buffer138. Additionally, the consumer processor 124 may detect a request froma virtual machine (e.g., VM 170A-B). Then, the consumer processor mayconsume the at least one memory entry from the ring buffer (block 430).For example, the consumer processor 124 may consume the memory entriesfrom the ring buffer 138. In an example, the consumer processor 124 mayconsume memory entries specified by the request.

The consumer processor may test a first value associated with arespective memory entry in a first slot of the plurality of slots, wherethe first slot is indicated by the head pointer (block 431). Forexample, the consumer processor 124 may test the slot indicated by thehead pointer 310 to determine if the slot includes a memory entry, suchas a data packet, or to determine if the slot is empty (e.g., includes aNULL value). Then, the consumer processor may retrieve the respectivememory entry from the first slot (block 432). For example, responsive toreceiving a valid value from the test, the consumer processor 124 mayretrieve the respective memory entry from the first slot. Then, theconsumer processor may advance the head pointer 310 to a second slot,while the location of the tail pointer 320 remains unchanged (block433). For example, the head pointer 310 and tail pointer 320 may bothstart at the same slot (e.g., first slot), and the consumer processor124 may advance the head pointer 310 to the next slot (e.g., secondslot) while maintaining the tail pointer 320 at the first slot. Then,the consume processor may determine whether the head pointer 310 hasadvanced a threshold quantity of slots (block 434). For example, thethreshold quantity of slots may be two, three, or more. The consumerprocessor may iterate blocks 431, 432, and 433 for each successive slotthat the head pointer 310 advances to until reaching the thresholdquantity of slots (e.g., two, three, or more slots). In an example, theconsumer processor 124 may test respective values of each successiveslot while the respective value of each successive slot in the ringbuffer 138 remains unchanged. Additionally, the consumer processor 124may stop testing slots responsive to receiving an invalid value from oneof the successive slots that the head pointer 310 is positioned at.Alternatively, if an invalid value is not received, the consumerprocessor 124 may stop testing responsive to retrieving a thresholdquantity of memory entries in the ring buffer 138 corresponding to athreshold quantity of slots that the head pointer 310 indicated whileadvancing along the ring buffer 138.

Optionally, the consumer processor may copy the respective memoryentries to a second memory. For example, the consumer processor 124 maycopy the respective memory entry associated with the first slot to thesecond memory, such as virtual machine memory 195A. In an example, theconsumer processor 124 may store the memory entries in a temporarymemory location before copying the memory entries to the second memory.The consumer processor 124 may copy memory entries to the second memoryin groups or batches, for example, after retrieving memory entries froma threshold quantity of slots. In another example, the consumerprocessor 124 may copy memory entries as they are retrieved, such thatthe memory entries are copied to the second memory iteratively alongwith the iterative testing, retrieving, and advancing described byblocks 431, 432, and 433 respectively.

In an example, the consumer processor may track a quantity of slots thatthe head pointer advances to. For example, the consumer processor 124may track a quantity of slots (e.g., two, three, or more slots) that thehead pointer 310 advances. In an example, the consumer processor 124 maycontinually track the quantity of slots that the head pointer 310advances via a counter, such as a lot counter that is maintained by theconsumer processor 124. Additionally, the consumer processor 124 maytrack the quantity of slots that the head pointer 310 advances bycomparing the position or location of the head pointer 310 and the tailpointer 320. The consumer processor 124 may track the quantity of slotsthat the head pointer advances to while the consumer processor 124 istesting and retrieving memory entries from successive slots. In anotherexample, the consumer processor 124 may track the quantity of slots atcertain times or intervals (e.g., after multiple iterations of testing,retrieving, and advancing occur).

If the threshold quantity is three slots, then the consumer processor124 may retrieve three memory entries from the ring buffer 138. If someof the slots include an invalid value, the consumer processor 124 mayretrieve less than three memory entries from the ring buffer 138. Forexample, the consumer processor 124 may advantageously retrieve lessthan the threshold quantity to increase efficiency and reduce downtimeassociated with waiting for additional packets to be produced by theproducer processor 128, which may cause undue delays when there is aslowdown of incoming network traffic. In an example, the consumerprocessor 124 may be paused or may wait before attempting to consumeadditional memory entries (e.g., the consumer processor may go to sleepbefore being woken up later to consume additional packets). For example,after testing a slot and receiving or reading an invalid value (e.g.,NULL value), a timer may start and count up to a predetermined timespan(e.g., 25 milliseconds, one second, 20 seconds) before the consumerprocessor 124 attempts to consume additional memory entries. In anotherexample, the consumer processor 124 may pause after testing a slot withan invalid value until it receives an additional request to copy memoryentries from the producer processor 128.

Then, the consumer processor may invalidate each slot between the headpointer and the tail pointer (block 440). For example, the consumerprocessor 124 may invalidate each slot between the head pointer 310 andthe tail pointer 320, for example, by overwriting the contents of theslot with an invalid value. Invalidating slots between the head pointer310 and the tail pointer 320 may include overwriting the memory entry ineach slot behind the head pointer 310 up to and including the slotindicated by the tail pointer 320. Additionally, the head pointer 310may be advanced after invalidating each of the slots. For example,invaliding slots between the head pointer 310 and the tail pointer 320may include invalidating the slots from the head pointer 310 up to thetail pointer 320 along with both slots indicated by the head pointer 310and tail pointer 320 respectively. In an example, the consumer processor124 may successively walk the ring buffer 138 backwards from the headpointer 310 (e.g., slot indicated by head pointer 310) to the tailpointer (e.g., slot indicated by the tail pointer 320). In an example,walking the ring buffer 138 backwards may include decrementing acounter, such as slot counter, while walking backwards. For example, theconsumer processor 124 may decrement the slot counter for each slot thatthe consumer processor 124 walks back to determine when it reaches thetail pointer 320. Invalidating slots between the head pointer 310 andthe tail pointer 320 may be done in any order.

Invaliding each slot between the head pointer 310 to the tail pointer320 may result in invalidating slots between the head pointer 310 andthe tail pointer 320 and the slot indicated by the tail pointer 320. Forexample, referring back to FIG. 3, invalidating each slot between thehead pointer 310C and the tail pointer 320A may result in invalidatingslots 210B-C (e.g., slots behind the slot indicated by head pointer310C). In an example, multiple invalidation steps may occur iterativelyafter advancing the head pointer 310 a threshold quantity of slots orreading an invalid value in a slot. For example, multiple invalidationsteps may occur to first invalidate slots 210C and then to invalidateslot 210B. In an example, invalidating each slot may include overwritingeach slot with an invalid value. For example, invalidating a slot mayinclude storing a NULL value, such as a “0” value in the slot.

By invalidating each slot in a batch all together instead of after eachretrieval, the producer processor 128 is advantageously held back fromaccessing the ring buffer 138 (e.g., cache line associated with slot inthe ring buffer 138), which reduces the frequency and/or occurrence of acache line bounce. Additionally, by consuming and invalidating athreshold quantity of slots (e.g., two or more slots), cache lineinvalidation messages are sent to the producer processor 128 less often,which advantageously reduces the frequency that the consumer processor124 is blocked while waiting for each of these cache line invalidationmessages to be received. For example, depending on the thresholdquantity used and the ring size, the producer processor 128 and consumerprocessor 124 may be accessing slots that are far enough away from eachother (e.g., in different cache lines) that neither is accessing thesame cache line at the same time. For example, a cache line may be 64bytes, and a slot may be 8 bytes, thus a ring buffer 138 may take upmultiple cache lines. If the producer processor 128 and the consumerprocessor 124 are producing and consuming memory entries in slots thatare positioned more than 8 slots away from each other in the aboveexample, the producer processor 128 and the consumer processor 124 maybe utilizing different cache lines. Accordingly, in this example, thethreshold quantity may be 8 or more so the respective cache lines usedby the producer processor 128 and the consumer processor 124 may notoverlap thereby avoiding or minimizing cache line bounces.

After the original slot is invalidated, a cache line may bounce betweenthe consumer processor 124 and the producer processor 128. For example,while the consumer processor 124 is accessing the ring buffer 138, theconsumer processor 124 may access the cache line for the ring buffer138. However, when the producer processor 128 needs to access the memoryfor the ring buffer 138, the producer processor 128 may request accessto the cache line to determine the current state of the ring buffer 138(e.g., what memory entries are present in the ring buffer 138), therebycausing a cache line bounce which may slow down the process offorwarding network traffic (e.g., data packets). In an example, a cacheline may be 64 bytes wide. When either the consumer processor 124 orproducer processor 128 needs to read or write a location in the ringbuffer 138, the consumer processor 124 and/or producer processor 128 maycheck for a corresponding entry in the cache. For example, the cache maycheck for the contents of the requested memory location (e.g., ringslot) in any cache lines that may contain the address.

FIGS. 5A, 5B, and 5C illustrate a flowchart of an example method 500 forsplit head invalidation for consumer batching in pointer rings inaccordance with an example embodiment of the present disclosure.Although the example method 500 is described with reference to theflowchart illustrated in FIGS. 5A, 5B, and 5C it will be appreciatedthat many other methods of performing the acts associated with themethod 500 may be used. For example, the order of some of the blocks maybe changed, certain blocks may be combined with other blocks, and someof the blocks described are optional. For example, a consumer processor124 (e.g., consumer CPU) and a producer processor 128 (e.g., producerCPU) may communicate with a ring buffer 138 and virtual machine memory195A to perform example method 500.

In the illustrated example, the producer CPU 128 may produce packetaddresses P_1 to P_4 to ring buffer 138 (blocks 502 and 504). Forexample, the producer CPU may produce packet addresses 0.x9000:0000,0x7000:0000, 0x3000:0000, and 0x2000:0000 corresponding to packetaddress P_1 to P_4 in four slots in the ring buffer 138 (e.g., firstslot to fourth slot). Then, the ring buffer or ring 138 receives packetaddresses P_1 to P_4 (block 506). For example, packet addresses P_1 toP_4 may be written in the ring buffer 138. In an example, packetaddresses P_1 to P_4 may be stored in the ring buffer 138 by writing thepacket address into the slot (e.g., replacing NULL value with packetaddress). In the illustrated example, the ring buffer 138 includes sixslots and stores packet addresses P_1 to P_4 in slot_1 to slot_4 whileslot_5 and slot_6 are NULL (block 508). In an example, a memory addressof “0” may represent a NULL value.

Then, the producer CPU 128 may request the consumer CPU 124 to copy allavailable packets to VM memory 195A (blocks 510 and 512). For example,the producer CPU 128 may forward incoming network traffic to a virtualmachine (e.g., VM 170A) by producing packet addresses for the incomingnetwork traffic to the ring buffer 138 and requesting the consumer CPU124 to copy the available packets (via the packet addresses) from thering buffer 138 to VM memory 195A. The consumer CPU may start consumingpackets at the head pointer 310 location (block 514). In an example, theconsumer CPU 124 may start consuming packets (e.g., packet addresses)after receiving the request from the producer CPU 128. Additionally, theconsumer CPU 124 may detect the request.

The consumer CPU 128 may test the first slot (block 516). For example,the consumer CPU 128 may test slot_1 to determine the value of thememory entry or packet address in the slot. In the illustrated example,the first slot includes a packet address for P_1 (e.g., 0x9000:0000)(block 518). For example, the consumer CPU 124 may read slot_1 whiletesting the first slot to determine that the first slot includes apacket address of 0x9000:0000. Then, the consumer CPU 124 may retrievethe packet address for P_1 (block 520). In an example, the consumer CPU124 may store the packet address for P_1 in a temporary memory locationbefore copying the associated packet to VM memory 195A.

After retrieving the packet address in the first slot, the consumer CPU124 may advance the head pointer 310 to the second slot (e.g., slot_2)and track advancement (block 522). In an example, the consumer CPU 124may track advancement by comparing the position of the head pointer 310to the tail pointer 320. In another example, the consumer CPU 124 mayimplement a counter to track advancement of the head pointer 310. Then,the consumer CPU may test the second slot (block 524). For example, theconsumer CPU 128 may test slot_2 to determine the value of the memoryentry or packet address in the slot. In the illustrated example, thesecond slot includes a packet address for P_2 (e.g., 0x7000:0000) (block526). For example, the consumer CPU 124 may read slot_2 while testingthe second slot to determine that the second slot includes a packetaddress of 0x7000:0000. Then, the consumer CPU 124 may retrieve thepacket address for P_2 (block 528). Similar to P_1, the packet addressfor P_2 may also be stored in a temporary memory location.

After retrieving the packet address for P_2, the consumer CPU 124 mayadvance the head pointer 310 to the third slot (e.g., slot_3) and trackadvancement of the head pointer 310 (block 530). Since the location ofthe tail pointer 320 has remained unchanged, the consumer CPU 124 maytrack advancement of the head pointer by comparing the position of thehead pointer 310 (e.g., positioned at slot_3) to the position of tailpointer 320 (e.g., positioned at slot_1) to determine that the headpointer 310 has advanced two slots. Then, the consumer CPU 124 maydetermine that the head pointer 310 has advanced a threshold quantity ofspaces or slots (block 532). For example, in the illustrated example,the threshold quantity of spaces or slots is two slots.

Then, as illustrated in FIG. 5B, the consumer CPU 124 may copy packetdata to VM memory 195A (blocks 534 and 536). For example, the consumerCPU 124 may copy the data packets associated with the packet addressesP_1 and P_2 to VM memory 195A. In an example, the packet data may becopied to VM memory 195A from a temporary memory location. Then, packetdata for P_1 and P_2 is copied to VM memory 195A (block 538). In anexample, packet data for P_1 and P_2 may be copied to VM memory 195A ina batch. Additionally, packet data for P_1 and P_2 may be copied to VMmemory 195A sequentially as each packet address is retrieved by theconsumer CPU 124. For example, packet data may be copied to VM memory195A before the head pointer 310 advances to the next slot.

After retrieving packet addresses from a threshold quantity of slots inthe ring buffer 138, the consumer CPU 124 may write or store a NULLvalue into the second slot (blocks 540 and 542). For example, theconsumer CPU 124 may walk the ring buffer 138 backwards from slot_3 toslot_1 and may invalidate slot_2 and slot_1 sequentially. Then, thesecond slot (e.g., slot_2) is overwritten with a NULL value (block 544).For example, slot_2 in the ring buffer 138 is invalidated, which enablesit to be used by the producer CPU 128 in the future for additionalnetwork traffic data packets. After invalidating the second slot, theconsumer CPU 124 may walk the ring buffer 138 backwards to the firstslot (block 546). For example, the consumer CPU 124 may execute a walkfunction to walk the ring buffer 138. Then, the consumer CPU 124 maywrite or store a NULL value into the first slot (blocks 548 and 550). Inan example, a memory address of “0” may represent a NULL value. Thesecond slot is overwritten with a NULL value (block 552). For example,slot_1 in the ring buffer 138 is invalidated, which enables slot_1 to beused by the producer CPU 128 for additional network traffic datapackets. Invaliding the slots in batches or groups advantageouslyimproves performance and throughput without increasing ring size,especially when a driver is highly optimized and copying is slower thanreceiving packets. After invalidating the first slot, the consumer CPU124 may advance the tail pointer to the third slot (block 554). Forexample, the consumer CPU 124 may advance the tail pointer 320 to thesame location as the head pointer 310. With the head pointer 310 and thetail pointer 320 located at the same slot, the consumer CPU 124 maytrack future advancement of the head pointer 310 to determine when thenext threshold quantity of slots has been reached.

The consumer CPU 124 may test the third slot (block 556). For example,the consumer CPU 128 may test slot_3 to determine the value of thememory entry or packet address in the slot. In the illustrated example,the third slot includes a packet address for P_3 (e.g., 0x3000:0000)(block 558). For example, the consumer CPU 124 may read slot_3 whiletesting the third slot to determine that the third slot includes apacket address of 0x3000:0000. Then, the consumer CPU 124 may retrievethe packet address for P_3 (block 560). As discussed above, the packetaddress (e.g., packet address for P_3) may be stored in a temporarymemory location by the consumer. After retrieving the packet addressfrom the third slot, the consumer CPU 124 may advance the head pointer310 to slot_4 and track advancement of the head pointer 310. Since thelocation of the tail pointer 320 was previously changed to the thirdslot (e.g., slot_3), the consumer CPU 124 may track advancement of thehead pointer by comparing the position of the head pointer 310 (e.g.,positioned at slot_4) to the position of tail pointer 320 (e.g.,positioned at slot_3) to determine that the head pointer 310 hasadvanced one slot.

Then, the consumer CPU 124 may test the fourth slot (block 564). Forexample, the consumer CPU 128 may test slot_4 to determine the value ofthe memory entry or packet address in the slot. In the illustratedexample, the fourth slot includes a packet address for P_4 (e.g.,0x2000:0000) (block 566). For example, the consumer CPU 124 may readslot_4 while testing the fourth slot to determine that the fourth slotincludes a packet address of 0x2000:0000. Then, the consumer CPU 124 mayretrieve the packet address for P_4 (block 568). In an example, theconsumer CPU 124 may store the packet address for P_4 in a temporarymemory location. In another example, the consumer CPU 124 may copy eachpacket to VM memory 195A immediately after retrieval.

As illustrated in FIG. 5C, after retrieving the packet address for P_4,the consumer CPU 124 may advance the head pointer 310 to the fifth slot(e.g., slot_5) and track advancement of the head pointer 310 (block570). For example, the consumer CPU 124 may determine that the headpointer 310 has advanced two slots (e.g., from slot_3 to slot_5). Then,the consumer CPU 124 may determine that the head pointer 310 hasadvanced a threshold quantity of spaces or slots (block 572). Forexample, in the illustrated example, the threshold quantity of spaces orslots is two slots. In an example, the threshold quantity of slots maybe four, five, ten, or more slots. Additionally, the threshold quantityof slots may be associated with the size of the ring buffer 138 and/orthe size of a cache line.

Then, the consumer CPU 124 may copy packet data to VM memory 195A(blocks 574 and 576). For example, the consumer CPU 124 may copy thedata packets associated with the packet addresses P_3 and P_4 to VMmemory 195A. In an example, the packet data may be copied to VM memory195A from a temporary memory location. Then, packet data for P_3 and P_4is copied to VM memory 195A (block 578). In an example, packet data forP_3 and P_4 may be copied to VM memory 195A in a batch. Additionally,packet data for P_3 and P_4 may be copied to VM memory 195A sequentiallyas each packet address is retrieved by the consumer CPU 124. Forexample, packet data may be copied to VM memory 195A before the headpointer 310 advances to the next slot.

After retrieving packet addresses from a threshold quantity of slots inthe ring buffer 138, the consumer CPU 124 may write or store a NULLvalue into the fourth slot (blocks 580 and 582). For example, theconsumer CPU 124 may walk the ring buffer 138 backwards from slot_5 toslot_3 and may invalidate slot_4 and slot_3 sequentially. Then, thefourth slot (e.g., slot_4) is overwritten with a NULL value (block 584).For example, slot_4 in the ring buffer 138 is invalidated, which enablesit to be used by the producer CPU 128 in the future for additionalnetwork traffic data packets. After invalidating the fourth slot, theconsumer CPU 124 may walk the ring buffer 138 backwards to the thirdslot (block 586). For example, the consumer CPU 124 may execute a walkfunction to walk the ring buffer 138. Then, the consumer CPU 124 maywrite or store a NULL value into the third slot (blocks 588 and 590). Inan example, a memory address of “0” may represent a NULL value. Thethird slot is overwritten with a NULL value (block 592). For example,slot_3 in the ring buffer 138 is invalidated, which enables slot_3 to beused by the producer CPU 128 for additional network traffic datapackets.

After invalidating the third slot, the consumer CPU 124 may advance thetail pointer to the fifth slot (block 592). For example, the consumerCPU 124 may advance the tail pointer 320 to the same location as thehead pointer 310. With the head pointer 310 and the tail pointer 320located at the same slot (e.g., slot_5), the consumer CPU 124 may trackfuture advancement of the head pointer 310 to determine when the nextthreshold quantity of slots has been reached. Then, the consumer CPU 124may test the fifth slot (block 594). For example, the consumer CPU 128may test slot_5 to determine the value of the memory entry or packetaddress in the slot. In the illustrated example, the fifth slot includesa NULL value or “0” value (block 596). For example, the consumer CPU 124may read slot_5 while testing the fifth slot to determine that the fifthslot includes a packet address of “0”. Upon receiving a NULL value, theconsumer CPU 124 may stop testing slots. For example, a NULL value mayindicate that the remainder of the ring buffer 138 is empty and thatthere are currently no additional packets to copy to VM memory 195. Inan example, the consumer processor 124 may be paused or may wait beforeattempting to consume additional memory entries. For example, aftertesting a slot and receiving or reading a NULL value, a timer may startand count up to a predetermined timespan (e.g., 25 milliseconds, onesecond, 20 seconds) before the consumer processor 124 attempts toconsume additional memory entries. In another example, the consumerprocessor 124 may pause after testing a slot with a NULL value until itreceives an additional request to copy memory entries from the producerprocessor 128.

At a later time, the producer CPU 128 may produce packets to empty orinvalid slots. In the illustrated example, the producer CPU 128 mayproduce packet addresses P_5, P_6, and P_7 to ring buffer 138 (blocks597 and 598). For example, the producer CPU may produce packet addresses0.x1000:0000, 0x8000:0000, and 0x6000:0000 corresponding to packetaddress P_5 to P_7 in three slots in the ring buffer 138 (e.g., slot_5,slot_6, and slot_1). Then, the ring buffer or ring 138 receives packetaddresses P_5 to P_7 and the ring 138 includes six slots and storespacket addresses P_5 to P_7 in slot_5, slot_6, and slot_1 respectivelywhile slot_1 to slot_4 are NULL (block 599). By previously invalidatingslot_1, the consumer CPU 124 advantageously made the slot available forpacket P_7. After producing additional packet addresses to the ring 138,the producer CPU 128 may send additional requests to the consumer CPU124 to copy available packets to VM memory 195A.

FIG. 6 is a block diagram of an example split head invalidation system600 according to an example embodiment of the present disclosure. Thesplit head invalidation system 600 includes a first memory 610 includinga ring buffer 612 having a plurality of slots 618A-C. The ring buffer612 may have any appropriate number of slots, which may vary dependingon application. The reverse slot invalidation system 600 may alsoinclude a second memory 620 and at least one processor 630A-B incommunication with the first memory 610. The at least one processor630A-B includes a consumer processor 630A and a producer processor 630B.The consumer processor 630A may be configured to maintain a head pointer632 and a tail pointer 634, detect a request 640 to copy at least onememory entry (e.g., memory entries 650A-B) from the ring buffer 612 tothe second memory 620, and consume the at least one memory entry (e.g.,memory entries 650A-B) from the ring buffer 612. Consuming memoryentries may include testing a respective value (values 680A-B)associated with a respective memory entry (e.g., memory entries 650A-B)in a first slot 625 (e.g., slot 618A), where the first slot 625 isindicated by the head pointer 632. Additionally, the consumer processor630A may test a respective memory entry (e.g., memory entry 650A) fromthe first slot 625 (e.g., slot 618A) responsive to receiving a validvalue from the test. The consumer processor 630A may also includeadvancing the head pointer 632 to a second slot 627 while the locationof the tail pointer 634 remains unchanged. Testing respective values,retrieving respective memory entries, and advancing the head pointer maybe iterated or repeated for each successive slot that the head pointeradvances to until reaching a threshold quantity of slots 660 (e.g., 2slots). Responsive to the quantity of slots (e.g., 2 slots) reaching thethreshold quantity of slots 660 (e.g., 2 slots), the consumer processor630A may be configured to invalidate each slot between the head pointer632′ and the tail pointer 634.

Instead of a cache line constantly bouncing between the producerprocessor 630B and consumer processor 630A, the consumer processor 630Aadvantageously retrieves and invalidates multiple memory entries 650from a threshold quantity (e.g., 2, 3, 8, 16) of slots 618 at a time,thereby allowing multiple memory entry operations before the cache linebounces to the producer processor 630B, which improves performance andthroughput without increasing the size of the ring buffer 612.

It will be appreciated that all of the disclosed methods and proceduresdescribed herein can be implemented using one or more computer programsor components. These components may be provided as a series of computerinstructions on any conventional computer readable medium ormachine-readable medium, including volatile or non-volatile memory, suchas RAM, ROM, flash memory, magnetic or optical disks, optical memory, orother storage media. The instructions may be provided as software orfirmware, and/or may be implemented in whole or in part in hardwarecomponents such as ASICs, FPGAs, DSPs or any other similar devices. Theinstructions may be configured to be executed by one or more processors,which when executing the series of computer instructions, performs orfacilitates the performance of all or part of the disclosed methods andprocedures.

It should be understood that various changes and modifications to theexample embodiments described herein will be apparent to those skilledin the art. Such changes and modifications can be made without departingfrom the spirit and scope of the present subject matter and withoutdiminishing its intended advantages. It is therefore intended that suchchanges and modifications be covered by the appended claims.

The invention is claimed as follows:
 1. A system comprising: a firstmemory including a ring buffer having a plurality of slots; a secondmemory; and at least one processor in communication with the firstmemory, wherein the at least one processor includes a consumer processorand a producer processor, and the consumer processor is configured to:maintain a head pointer and a tail pointer, detect a request to copy atleast one memory entry from the ring buffer to the second memory,consume the at least one memory entry from the ring buffer, including:(i) test a respective value associated with a respective memory entry ina first slot of the plurality of slots, wherein the first slot isindicated by the head pointer, (ii) responsive to receiving a validvalue from the test, retrieve the respective memory entry from the firstslot, (iii) advance the head pointer to a second slot while the locationof the tail pointer remains unchanged, and (iv) iterating (i), (ii), and(iii) for each successive slot that the head pointer advances to untilreaching a threshold quantity of slots, and responsive to a quantity ofslots reaching the threshold quantity of slots, invalidate each slotbetween the head pointer and the tail pointer.
 2. The system of claim 1,wherein the consumer processor is configured to advance the tail pointerto the location of the head pointer after invalidating each slot betweenthe head pointer and the tail pointer.
 3. The system of claim 1, whereinthe head pointer and the tail pointer are maintained in the same cacheline.
 4. The system of claim 1, wherein the head pointer and the tailpointer are indices.
 5. The system of claim 1, wherein the thresholdquantity of slots occupy an integer number of cache lines.
 6. The systemof claim 1, wherein the head pointer and the tail pointer start at thesame slot.
 7. The system of claim 1, further comprising a counter,wherein the consumer processor is configured to track the quantity ofslots with the counter.
 8. The system of claim 1, wherein the consumerprocessor compares the location of the head pointer and the tail pointerto track the quantity of slots that the head pointer advances.
 9. Thesystem of claim 1, wherein the consumer processor and the producerprocessor are on different cores of the same physical processor.
 10. Thesystem of claim 1, wherein the consumer processor and the producerprocessor are different physical processors.
 11. The system of claim 1,wherein the at least one memory entry is at least one packet address andthe second memory is a virtual machine memory.
 12. A method comprising:maintaining, by a consumer processor, a head pointer and a tail pointer;detecting, by a consumer processor, a request to copy at least onememory entry from a ring buffer having a plurality of slots; consuming,by the consumer processor, the at least one memory entry from the ringbuffer, including: (i) testing, by the consumer processor, a first valueassociated with a respective memory entry in a first slot of theplurality of slots, wherein the first slot is indicated by the headpointer, (ii) responsive to receiving a valid value from the test,retrieve, by the consumer processor, the respective memory entry fromthe first slot, (iii) advancing, by the consumer processor, the headpointer to a second slot while the location of the tail pointer remainsunchanged, and (iv) iterating, by the consumer processor, (i), (ii), and(iii) for each successive slot that the head pointer advances to untilreaching a threshold quantity of slots; and responsive to a quantity ofslots reaching the threshold quantity of slots, invalidating, by theconsumer processor, each slot between the head pointer and the tailpointer.
 13. The method of claim 12, further comprising tracking, by theconsumer processor, the quantity of slots the head pointer advances to,wherein tracking the quantity of slots used includes comparing the headpointer and the tail pointer.
 14. The method of claim 12, furthercomprising copying, by the consumer processor, the respective memoryentries to a second memory.
 15. The method of claim 12, furthercomprising advancing, by the consumer processor, the tail pointer to thelocation of the head pointer.
 16. The method of claim 12, wherein thehead pointer and the tail pointer are maintained in the same cache linein the memory.
 17. The method of claim 12, wherein the consumerprocessor detects a request from one of the virtual machine and theproducer processor.
 18. The method of claim 12, wherein invalidatingeach slot between the head pointer and the tail pointer includesinvalidating each slot behind a third slot indicated by the head pointerto the first slot indicated by the tail pointer, which includesinvalidating the first slot indicated by the tail pointer.
 19. Themethod of claim 12, wherein the consumer processor advances the headpointer to a third slot and the consumer processor invalidates thesecond slot and the first slot, wherein the first slot, the second slot,and the third slot are successive slots in the ring buffer.
 20. Anon-transitory machine-readable medium storing code, which when executedby a consumer processor, is configured to: maintain a head pointer and atail pointer; detect a request to copy a at least one memory entry froma ring buffer having a plurality of slots; consume the at least onememory entry from the ring buffer, including: (i) test a first valueassociated with a respective memory entry in a first slot of theplurality of slots, wherein the first slot is indicated by the headpointer, (ii) responsive to receiving a valid value from the test,retrieve the respective memory entry from the first slot, (iii) advancethe head pointer to a second slot while the location of the tail pointerremains unchanged, and (iv) iterate (i), (ii), and (iii) for eachsuccessive slot that the head pointer advances to until reaching athreshold quantity of slots; and responsive to the quantity of slotsreaching the threshold quantity of slots, invalidate each slot betweenthe head pointer and the tail pointer.